Papers by Kavin R V
Breaking Token Into Concepts: Exploring Extreme Compression in Token Representation Via Compositional Shared Semantics (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Standard language models employ unique, monolithic embeddings for each token, limiting their ability to capture multifaceted meanings. |
| Approach: | They propose a compositional structure that accumulates diverse semantic facets for tokens . they apply this representational scheme to standard transformer architectures and a biomedical domain benchmark . |
| Outcome: | The proposed representational scheme achieves extreme compression in embedding parameters while maintaining >95% task performance relative to the base model. |